Papers with multilingual training
CLEAR: Cross-Lingual Enhancement in Retrieval via Reverse-training (2026.acl-long)
Copied to clipboard
| Challenge: | Existing multilingual embedding models often struggle to capture cross-lingual alignment during training. |
| Approach: | They propose a novel loss function that leverages an English passage as a bridge to strengthen alignments between target language and English. |
| Outcome: | The proposed model improves retrieval performance across cross-lingual scenarios while minimizing performance degradation in English. |
Exploring Cross-Lingual Voice Conversion Methods for Anonymizing Low-Resource Text-to-Speech (2026.eacl-short)
Copied to clipboard
| Challenge: | a growing number of speech synthesis systems clone a person's voice, a new study finds . a variety of voice conversion techniques can mask speaker identities in low-resource text-to-speech systems. |
| Approach: | They compare voice conversion techniques to mask speaker identities in text-to-speech systems . they build and evaluate speaker-anonymized systems for two Canadian Indigenous languages . |
| Outcome: | The proposed methods are compared with other approaches for using voice conversion to mask speaker identities in low-resource text-to-speech systems. |
English as Defense Proxy: Mitigating Multilingual Jailbreak via Eliciting English Safety Knowledge (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models excel in many tasks, but their safety guarantees vary by language. |
| Approach: | They propose a unified approach that leverages English as a universal safety anchor. |
| Outcome: | The proposed approach leverages English as defense proxy (E-Proxy) to transfer safety knowledge across languages. |
IndicBART: A Pre-trained Model for Indic Natural Language Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | IndicBART is a multilingual, sequence-to-sequence pre-trained model focusing on 11 Indic languages and English. |
| Approach: | They present a multilingual sequence-to-sequence pre-trained model for Indic languages . they evaluate it on two NLG tasks: Neural Machine Translation and extreme summarization . |
| Outcome: | The proposed model performs well on low-resource translation scenarios . Script sharing, multilingual training, and better utilization contribute to the performance. |
Competence-based Curriculum Learning for Multilingual Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual machine translation models face an imbalance problem due to the different learning competencies of different languages. |
| Approach: | They propose Competence-based Curriculum Learning for Multilingual Machine Translation, named CCL-M, to help schedule the high resource languages and low resource languages. |
| Outcome: | The proposed approach achieves a steady and significant performance gain compared to the previous state-of-the-art approach on the TED talks dataset. |
Breaking Down Multilingual Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Multilingual training is an essential ingredient in machine translation systems . but it has different effects in different multilingual settings, such as many-to-one, one-tomany and many- to-many learning . |
| Approach: | They compare multilingual training settings with encoders and decoders initialized by multilingual learning . they find important attention heads for each language pair and compare their correlations during inference . |
| Outcome: | The proposed models outperform the best models for high-resource languages and one-to-many models for low-resourced languages. |
Revisiting non-English Text Simplification: A Unified Multilingual Benchmark (2023.acl-long)
Copied to clipboard
| Challenge: | Recent advances in English automatic text simplification have pushed the frontier of multilingual text simulating. |
| Approach: | They propose to use multilingual evaluation benchmarks to evaluate multilingual text simplification models in English and other languages. |
| Outcome: | The proposed benchmark outperforms pre-trained models in Russian in zero-shot cross-lingual transfer to low-resource languages. |
75 Languages, 1 Model: Parsing Universal Dependencies Universally (D19-1)
Copied to clipboard
| Challenge: | UDify is a multilingual multi-task model that can predict universal part-of-speech, morphological features, lemmas, and dependency trees. |
| Approach: | They evaluate UDify, a multilingual multi-task model capable of predicting universal part-of-speech, morphological features, lemmas, and dependency trees simultaneously for all 124 Universal Dependencies treebanks across 75 languages. |
| Outcome: | The proposed model can predict universal part-of-speech, morphological features, lemmas, and dependency trees for all 124 treebanks across 75 languages. |
GradSim: Gradient-Based Language Grouping for Effective Multilingual Training (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that not all languages positively influence each other . multilingual training can help in those cases by sharing knowledge across languages . |
| Approach: | They propose a gradient similarity-based language grouping method for multilingual training that is better correlated with cross-lingual model performance. |
| Outcome: | The proposed method leads to the largest performance gains on a multilingual dataset and is better correlated with cross-lingual model performance. |
CONGRAD: Conflicting Gradient Filtering for Multilingual Preference Alignment (2026.eacl-long)
Copied to clipboard
Jiangnan Li, Thuy-Trang Vu, Christian Herold, Amirhossein Tebbifakhr, Shahram Khadivi, Gholamreza Haffari
| Challenge: | Naive joint training of large language models can suffer from negative interference. |
| Approach: | They propose a filtering method that aggregates cross-lingually beneficial gradients and filters for those with high cross-linguistic affinity. |
| Outcome: | The proposed method outperforms baselines in both seen and unseen languages with minimal alignment tax. |
Zero-shot Cross-lingual Alignment for Embedding Initialization (2024.findings-acl)
Copied to clipboard
| Challenge: | CrossInit initializes embeddings into similar geometrical structures across languages in unsupervised manner. |
| Approach: | They propose a method that initializes embeddings into similar geometrical structures across languages in an unsupervised manner. |
| Outcome: | The proposed method demostrates similar patterns in low-resource and dissimilar languages. |
PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification (D19-1)
Copied to clipboard
| Challenge: | Existing work on adversarial data generation focuses on English . Existing multilingual datasets show effectiveness of deep, multilingual pre-training . |
| Approach: | They propose a dataset of 23,659 human translated PAWS evaluation pairs in six languages . they show the effectiveness of deep, multilingual pre-training while leaving considerable headroom . |
| Outcome: | The proposed model shows that multilingual training and evaluation regimes are more accurate than previous models. |
XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages (2021.findings-acl)
Copied to clipboard
Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M. Sohel Rahman, Rifat Shahriyar
| Challenge: | XL-Sum dataset covers 44 languages ranging from low to high-resource . Xl-SUM is highly abstractive, concise, and of high quality . |
| Approach: | They present a dataset comprising 1 million professionally annotated article-summary pairs from BBC . they fine-tune a pretrained multilingual model with XL-Sum and experiment on multilingual and lowresource tasks. |
| Outcome: | The proposed dataset is highly abstractive, concise, and of high quality . it shows higher scores on 10 languages than similar datasets compared to monolingual ones . |
The SADID Evaluation Datasets for Low-Resource Spoken Language Machine Translation of Arabic Dialects (2020.coling-main)
Copied to clipboard
| Challenge: | Low-resource Machine Translation (LRT) models are still lagging behind on low-resourced language pairs due to the scarcity of parallel training data. |
| Approach: | They introduce benchmark datasets for Arabic and its dialects to examine their properties . they bootstrap existing parallel sentences and complement this with multilingual training . |
| Outcome: | The proposed method bootstraps existing parallel sentences and complements multilingual training to achieve strong baselines. |
MAPO: Advancing Multilingual Reasoning through Multilingual-Alignment-as-Preference Optimization (2024.acl-long)
Copied to clipboard
| Challenge: | Existing models exhibit inconsistent reasoning abilities across different languages . existing models lack consistency across languages due to imbalance of training data . |
| Approach: | They propose a multilingual alignment-as-preference optimization framework to align reasoning processes in other languages with the dominant language. |
| Outcome: | The proposed framework improves multilingual reasoning across languages on three benchmarks. |
Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models (2024.emnlp-main)
Copied to clipboard
Terra Blevins, Tomasz Limisiewicz, Suchin Gururangan, Margaret Li, Hila Gonen, Noah Smith, Luke Zettlemoyer
| Challenge: | Multilingual language models often underperform monolingual ones due to inter-language competition for model parameters. |
| Approach: | They propose Cross-lingual Expert Language Models (X-ELM) which mitigates inter-language competition by independently training language models on subsets of the multilingual corpus. |
| Outcome: | The proposed model outperforms jointly trained multilingual models across all 16 considered languages and transfer the gains to downstream tasks. |
Automatic Speech Recognition for Uyghur through Multilingual Acoustic Modeling (2020.lrec-1)
Copied to clipboard
| Challenge: | Low-resource languages suffer from lower performance of Automatic Speech Recognition (ASR) due to the lack of data. |
| Approach: | They propose to use Turkish as donor language to train acoustic models using multilingual training to achieve more context coverage. |
| Outcome: | The proposed system performs better with multilingual training for the under-resourced Uyghur language. |
XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations (2023.acl-long)
Copied to clipboard
| Challenge: | Existing models for cross-lingual semantic parsing are not able to perform tasks on a wide range of datasets. |
| Approach: | They propose a benchmark for cross-lingual semantic parsing that uses 22 natural languages and 8 meaning representations to translate queries into MRs. |
| Outcome: | The proposed benchmarks cover 22 natural languages and 8 meaning representations on 164 domains and 5 tasks covering a wide range of multilingual language models. |
Sounding vs. Being an Expert: Disentangling Authority, Register and Cultural Impact in Sycophantic LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models exhibit sycophancy, a tendency to align with user assertions even when they conflict with factual correctness. |
| Approach: | They propose an adversarial evaluation framework that isolates two drivers of credibility: explicit authority (credentials) and implicit authority (linguistic register). |
| Outcome: | The proposed framework disentangles two drivers of credibility: explicit authority (credentials) and implicit authority (linguistic register). |
The Impact of Language Mixing on Bilingual LLM Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show multilingual speakers intentionally switch languages during reasoning . enforcing monolingual decoding reduces accuracy by 5.6 percentage points . |
| Approach: | They find that multilingual speakers intentionally switch languages during reasoning . enforcing monolingual decoding reduces accuracy by 5.6 percentage points . authors suggest that language mixing is not merely a byproduct of multilingual training . |
| Outcome: | The proposed model can be used to predict whether a language switch would benefit or harm reasoning. |
Multilinguality Does not Make Sense: Investigating Factors Behind Zero-Shot Cross-Lingual Transfer in Sense-Aware Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual transfer allows models to perform tasks in languages unseen during training and is often assumed to benefit from increased multilinguality. |
| Approach: | They challenge this assumption by analyzing polysemy disambiguation and lexical semantic change in 28 languages and using confounding factors to account for perceived advantages. |
| Outcome: | The proposed models and benchmarks are compared across 28 languages and show that multilingual training is neither necessary nor beneficial for effective transfer. |